feat(shard-distributor): add SpectatorPeerChooser for shard-aware routing #7478
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jakobht
requested review from
3vilhamster,
Shaddoll,
davidporter-id-au,
demirkayaender,
dkrotx,
neil-xie,
sankari165,
shijiesheng and
taylanisikdemir
as code owners
jakobht
force-pushed
the
add-spectator-peer-chooser
branch
3 times, most recently
from
fbee9e9 to
642ad52
Compare
Previously GetShardOwner only returned the executor ID string. Now it returns an ExecutorOwnership struct containing both the executor ID and the full metadata map. This allows callers to access additional executor information like the gRPC address needed for peer routing. Changes: - Add ExecutorOwnership struct with ExecutorID and Metadata fields - Update GetShardOwner signature to return *ExecutorOwnership - Update spectatorImpl to build and return ownership info - Update tests to verify metadata is included in responses Signed-off-by: Jakob Haahr Taankvist <[email protected]>
Use BlockUntil() to properly synchronize with the fake clock before advancing time. This prevents the test from hanging while waiting for the goroutine to process the time advancement. Signed-off-by: Jakob Haahr Taankvist <[email protected]>
jakobht
force-pushed
the
add-spectator-peer-chooser
branch
from
642ad52 to
1597904
Compare
Encapsulate the spectators map in a struct to hide implementation details and make it easier to extend in the future without breaking the API. Signed-off-by: Jakob Haahr Taankvist <[email protected]>
Implement a YARPC peer chooser that routes requests to the correct
executor based on shard ownership. This is the shard distributor
equivalent of Cadence's RingpopPeerChooser.
Flow:
1. Client calls RPC with yarpc.WithShardKey("shard-key")
2. Chooser queries Spectator for shard owner
3. Extracts grpc_address from owner metadata
4. Creates/reuses peer for that address
5. Returns peer to YARPC for connection
The peer chooser maintains a cache of peers and handles concurrent
access safely. It uses the x-shard-distributor-namespace header to
determine which namespace's spectator to query.
Dependencies:
- Requires spectator GetShardOwner to return metadata (see previous commit)
Signed-off-by: Jakob Haahr Taankvist <[email protected]>
Signed-off-by: Jakob Haahr Taankvist <[email protected]>
The tests have dependency issues with mock generation that need to be resolved separately. The peer chooser implementation is complete and functional. Signed-off-by: Jakob Haahr Taankvist <[email protected]>
Tests cover: - Success path with peer creation - Peer reuse on subsequent calls - Error cases (missing shard key, namespace header, spectator not found) - Lifecycle methods (Start, Stop, IsRunning) - SetSpectators method Signed-off-by: Jakob Haahr Taankvist <[email protected]>
jakobht
force-pushed
the
add-spectator-peer-chooser
branch
from
1597904 to
f4bb653
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR depends on #7476 (spectator return metadata) being merged first.
What changed?
Added
SpectatorPeerChooserthat implements YARPC's peer.Chooser interface to route requests to the correct executor based on shard ownership.Why?
Enable executor-to-executor communication in the shard distributor canary system. The peer chooser queries the Spectator to find which executor owns a shard, then routes requests to that executor's gRPC address.
How did you test it?
Unit tests
Potential risks
None
Release notes
Documentation Changes